Skip to content

Support precompiled shaders and excluding the shader compiler#1598

Merged
bghgary merged 15 commits intoBabylonJS:masterfrom
bghgary:support-no-shader-compiler
Feb 7, 2026
Merged

Support precompiled shaders and excluding the shader compiler#1598
bghgary merged 15 commits intoBabylonJS:masterfrom
bghgary:support-no-shader-compiler

Conversation

@bghgary
Copy link
Contributor

@bghgary bghgary commented Feb 4, 2026

  • Introduces ShaderTool: A command-line tool for precompiling GLSL shaders offline
  • Refactors shader compilation into a separate ShaderCompiler plugin with optional runtime inclusion
  • Refactors shader caching into a separate ShaderCache plugin with improved hash-based storage
  • Adds ShaderProvider abstraction in NativeEngine to support both runtime compilation and precompiled shaders
  • Moves BgfxShaderInfo to Core/Graphics for shared access across components

@bghgary bghgary requested a review from Copilot February 5, 2026 01:21
@bghgary bghgary changed the title Support precompiled shaders such that the runtime no longer includes the shader compiler Support precompiled shaders such that it's possible to not include the shader compiler Feb 5, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds support for precompiled shaders to allow removing the shader compiler from the runtime. The changes introduce three new plugin components and refactor the existing shader compilation and caching infrastructure.

Changes:

  • Introduces ShaderTool: A command-line tool for precompiling GLSL shaders offline
  • Refactors shader compilation into a separate ShaderCompiler plugin with optional runtime inclusion
  • Refactors shader caching into a separate ShaderCache plugin with improved hash-based storage
  • Adds ShaderProvider abstraction in NativeEngine to support both runtime compilation and precompiled shaders
  • Moves BgfxShaderInfo to Core/Graphics for shared access across components

Reviewed changes

Copilot reviewed 48 out of 54 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
Plugins/ShaderTool/* New command-line tool for offline shader compilation
Plugins/ShaderCompiler/* Extracted shader compiler as optional plugin with public/internal APIs
Plugins/ShaderCache/* Refactored shader cache with xxHash-based hashing and improved serialization
Plugins/NativeEngine/* Refactored to use ShaderProvider abstraction supporting both compilation modes
Plugins/ShaderProvider/Source/ShaderProvider.cpp Incomplete/corrupted file (critical issue)
Core/Graphics/* Added BgfxShaderInfo structure and updated include paths
Apps/* Updated tests and Playground app to use new ShaderCache API
CMakeLists.txt files Added build options for new plugins and updated dependencies
Comments suppressed due to low confidence (2)

Plugins/ShaderCompiler/Source/ShaderCompilerD3D.cpp:12

  • The file uses std::array on line 18 but does not include or <arcana/experimental/array.h>. This will likely cause a compilation error. The other platform-specific shader compiler files (Metal, OpenGL, Vulkan) include <arcana/experimental/array.h> for this purpose.
    Plugins/ShaderCompiler/Source/ShaderCompilerD3D.cpp:12
  • The file uses gsl::narrow_cast, gsl::make_span, and gsl::span but does not include the GSL header. While ShaderCompilerCommon.h includes <gsl/gsl>, this file includes ShaderCompilerCommon.h after other headers. If ShaderCompilerCommon.h doesn't provide these definitions (for instance if it's protected by include guards that were already satisfied), this could cause compilation errors. Consider adding an explicit #include <gsl/gsl> directive or ensuring the include order is correct.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 51 out of 55 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (7)

Plugins/ShaderCache/Source/ShaderCacheImpl.cpp:33

  • The NormalizeLineEndings function creates a new string copy every time GetShader is called, even if the source doesn't contain any '\r' characters. Consider optimizing this by first checking if normalization is needed (i.e., if the source contains '\r'), and only creating a copy if necessary. This would avoid unnecessary allocations for the common case of LF-only line endings.
    Plugins/ShaderCompiler/Source/ShaderCompilerMetal.cpp:83
  • The parameter type for CompileInternal should be std::string_view (as declared in the header), not std::string by value. This is inconsistent with the declaration in ShaderCompilerInternal.h line 19.
    Plugins/ShaderCache/Source/ShaderCacheImpl.cpp:53
  • The TODO comment mentions endianness concerns for hash values. While the hash values themselves don't need to match across different platforms (since each platform compiles its own cache), the issue could arise if a cache file created on a big-endian system is used on a little-endian system. Consider documenting that shader cache files are platform-specific and should not be shared across different architectures, or implement endianness handling if cross-platform cache files are intended.
    Plugins/ShaderCompiler/Source/ShaderCompilerTraversers.h:73
  • The function signature was changed from std::unordered_map to std::map for the vertexAttributeRenaming parameter. However, the call sites in ShaderCompilerMetal.cpp line 106, ShaderCompilerOpenGL.cpp line 86, and ShaderCompilerVulkan.cpp line 81 still declare local variables as std::unordered_map<std::string, std::string>. This type mismatch will cause compilation errors. Those call sites need to be updated to use std::map as well.
    Plugins/ShaderCompiler/Source/ShaderCompilerOpenGL.cpp:64
  • The parameter type for CompileInternal in the header is std::string_view, but the implementation in OpenGL, Metal, and D3D takes std::string by value. This creates an inconsistency where Vulkan uses std::string (by value), while D3D uses std::string_view in the function signature. The implementation should match the header declaration which uses std::string_view.
    Plugins/ShaderCompiler/Source/ShaderCompilerVulkan.cpp:58
  • The parameter type for CompileInternal in ShaderCompilerVulkan.cpp uses std::string (by value), while the declaration in ShaderCompilerInternal.h line 19 uses std::string_view. This needs to be changed to std::string_view to match the header declaration and avoid unnecessary copying.
    Plugins/ShaderCache/Source/ShaderCacheImpl.cpp:137
  • The AddShader method does not normalize line endings when creating the ShaderHash, but GetShader does normalize them. This creates an inconsistency where shaders added via AddShader may not be found by GetShader if there are Windows (CRLF) vs Unix (LF) line ending differences. The shader sources should be normalized in AddShader as well to ensure consistency.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 52 out of 56 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (5)

Plugins/ShaderCompiler/Source/ShaderCompilerD3D.cpp:12

  • Missing include for std::array. The code uses std::array<const char*, 1> on line 18, but there is no #include <array> directive in the file. While this might compile due to transitive includes from other headers, it violates the self-contained header principle and could break if those other headers change. Add #include <array> to the includes section.
    Plugins/ShaderCompiler/Source/ShaderCompilerVulkan.cpp:11
  • Missing include for std::array. The code uses std::array<const char*, 1> on line 17, but there is no #include <array> directive in the file. While this might compile due to transitive includes from other headers (specifically arcana/experimental/array.h on line 5), it violates the self-contained header principle and could break if those other headers change. Add #include <array> to the includes section.
    Plugins/ShaderCache/Source/ShaderCacheImpl.cpp:32
  • The NormalizeLineEndings function doesn't pre-allocate memory for the result string, which could lead to multiple reallocations during string building. For better performance, consider calling result.reserve(source.size()) before the loop to pre-allocate memory. This is especially important since shader source code can be quite large, and the normalized result will typically be very close in size to the input (only slightly smaller due to removed carriage returns).
    Apps/Playground/Win32/App.cpp:94
  • There's a potential issue with error handling in the shader cache save operation. If Babylon::Plugins::ShaderCache::Save(stream) throws an exception, the Disable() call on line 93 will not be executed, leaving the shader cache in an enabled state. This could lead to resource leaks or inconsistent state on subsequent calls. Consider using RAII or a try-catch block to ensure Disable() is always called:
if (Babylon::Plugins::ShaderCache::IsEnabled())
{
    try {
        std::ofstream stream{shaderCachePath, std::ios::binary};
        Babylon::Plugins::ShaderCache::Save(stream);
    }
    catch (...) {
        Babylon::Plugins::ShaderCache::Disable();
        throw;
    }
    Babylon::Plugins::ShaderCache::Disable();
}
            width,
            height,
            [](const char* message) {
                std::ostringstream ss{};
                ss << message << std::endl;
                OutputDebugStringA(ss.str().data());
                std::cout << ss.str();

Plugins/ShaderCompiler/Source/ShaderCompilerOpenGL.cpp:11

  • Missing include for std::array. The code uses std::array<const char*, 1> on line 16, but there is no #include <array> directive in the file. While this might compile due to transitive includes from other headers (specifically arcana/experimental/array.h on line 5), it violates the self-contained header principle and could break if those other headers change. Add #include <array> to the includes section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@bghgary bghgary changed the title Support precompiled shaders such that it's possible to not include the shader compiler Support precompiled shaders and excluding the shader compiler Feb 6, 2026
@bghgary bghgary marked this pull request as ready for review February 6, 2026 18:50
@bghgary bghgary merged commit ec95b26 into BabylonJS:master Feb 7, 2026
25 checks passed
@bghgary bghgary deleted the support-no-shader-compiler branch February 7, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants